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This paper examines how collaborative on-line databases can 
be used to meet the need for individualized instruction for academic 
vocabulary learners intending to do university work in English. It is argued 
that effective teaching and course design for preparing non-native English 
speaking students for study at English language universities must be highly 
specialized if it is to be effective. For example, the Chinese speaking 
student preparing for a computer science graduate program in Canada needs to 
know a very different sort of specialized vocabulary than the French student 
preparing for a program in theater arts in Canada. The challenge for the 
course designer is to build a course that both of these students (and many 
others) can profit from. The solution proposed is to put technology tools in 
the hands of the learners so that they can construct the courses they need 
for themselves. This paper describes in detail how the authors tested an 
experimental English-as-a-Second-Language (ESL) vocabulary course for 
academic learners at Concordia University in Montreal, Quebec. The course 
design questions the authors set out to answer are the following: How can we 
ensure that a vocabulary course offers academic learners of varying first 
language backgrounds, second language proficiency levels, and academic 
objectives a significant opportunity to focus on the words they need to know? 
Can one course do all this? It is concluded that a collaborative database is 
a valuable tool for such learners. The technology is readily available and 
the learners have shown that they are willing and able to use the 
computerized, online tools developed for them for this purpose. (Contains 27 
references.) (KFT) 
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Most educators would agree that one of the most exciting 
aspects of using computerized resources for second 
language acquisition is the potential of these resources for 
meeting the very diverse needs of individual learners 
efficiently and effectively. We would argue that nowhere is 
the need for individualized instruction greater than in the 
area of academic vocabulary for learners intending to do 
university work in English. The basis for this claim is the 
fact that many university-bound ESL learners already know 
the core vocabulary of English; that is, they have progressed 
beyond the point where an all-purpose course in the most 
basic thousand words would be equally beneficial for all. As 
proficiency and vocabulary size increase, learners vary 
hugely in what they know and what they need to learn 
(Nation, 1990). Thus, the Chinese-speaking learner arriving 
in Canada to study computer science may already know - as 
a result of study in his home country - many words that will 
be useful for doing academic work in English. However, he 
is likely to know a very different set of items from the 
student seated next to him in an ESL class, for example a 
Francophone learner from Quebec intending to study stage 
design. It is also clear that the words these two students 
will want to study to prepare for courses in their majors will 
be quite different. The challenge for the course designer is 
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to build a course that both of these students (and many 
others) can profit from. Our proposed answer to the 
challenge is to put technology tools in the hands of learners 
so that they can construct the courses they need for 
themselves. In this paper we describe how we tested this 
idea in an experimental ESL vocabulaiy course for academic 
learners at Concordia University in Montreal. 

The course-design questions the authors set out to answer 
can be stated as follows. How can we ensure that a 
vocabulary course offers academic learners of vaiying LI 
backgrounds, L2 proficiency levels, and academic objectives 
a significant opportunity to focus on the words they need to 
know? Can one course address the needs of students 
preparing for study in content domains as varied as 
computer science and stage design? 

In addition to tailoring a course to meet students' 
vocabulary needs as effectively as possible, we were 
interested in investigating the role of technology in 
providing individualized instruction, and we wished to test a 
number of research claims about instructed vocabulary 
learning as well. The following section details the principles 
that guided the design of the course and outlines our 
research agenda. 



Designing the course 



The guidelines for the design of the course were as follows. 

Start with reading as a source of new vocabulary. 

• Provide technology tools for students to create the 
course they need. 

• Recognize that studying domain-specific vocabulary is 
important. 

• Recognize that knowing sub-technical academic 
vocabulary is crucial. 

• Raise awareness of proven word learning strategies. 
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• Challenge learners to study hundreds of new words. 

Start with reading 

There are at least four good reasons for focusing an 
academic vocabulary course on reading. First, although 
umversity- bound students need vocabulary knowledge to be 
fu 6 .f. peak and write in content courses, it is clear that 
the ability to read and understand course content as 
presented in textbooks is central. People who know more 
words understand more of what they read than people who 
know fewer. Indeed, reading comprehension is so closely 
associated with vocabulary knowledge that test designers 

i*Q 07 | had e diffiCUlly in distin s uishin g between the two (Read, 
1997). Secondly, receptive knowledge represents the 
starting point of the word learning process. Generally, LI 
and L2 learners alike begin to feel they know a new word 
when they can recognize its meaning when they read (or 
hear) it, while more active knowledge such as the ability to 
produce a fully correct definition of the word or to use it 
accurately in an original sentence tends to come later 
(Wesche & Panbakht, 1996). 

Thirdly, analyses of large corpora of written and spoken 
language indicate that written texts are much more likely to 
contain words that would be unfamiliar to intermediate ESL 
learners than spoken texts. Spoken discourse makes heavy 
use of common words - words that most intermediate ESL 
learners would probably already know - and rarely presents 
items outside a list of the 2000 most frequent words of 
Enghsh (West & Stanovich, 1991). It is clear that requiring 
students to read widely is a good way of ensuring they have 
many opportunities to meet new words beyond this most 
basic level. Finally, reading passages are natural ready- 
made learning materials for vocabulary acquisition because 
they present unfamiliar words in authentic sentence and 
chscourse contexts. These contexts can provide the learner 
with valuable grammar information, rich associative links to 
other words, and, importantly, useful clues to meaning 
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In implementing an individualized course with reading at its 
centre, the course designers felt it was important to give the 
students a role in selecting the texts that would serve as the 
basis for their vocabulary learning. Thus, instead of 
prescribing a core text, we required students to buy (or read 
on-line) a quality newspaper and read any two articles of 
their choice each week. The newspaper that we chose for 
this purpose was the Focus section of the weekend issue of 
the Toronto Globe and Mail This weekly supplement 
presents well-written essays on a variety of topics written in 
a style that can be termed academic. Each week students 
were required to prepare brief oral or written summaries of 
the two articles they had chosen. They were also expected to 
look up the meanings of new words they encountered in 
dictionaries. 

Provide technology tools for students to create the 
course they need 

The next question that the course designers faced was how 
we might use the valuable information gleaned in individual 
word quests to build a student-generated vocabulary course 
that all could access easily. While we recognized that not 
every student would be interested in each of every other 
student's dictionary findings, we reasoned that each 
student would belong to a number of constituencies within 
the class that had common vocabulary needs. That is, a 
new word encountered by one Francophone learner in a 
newspaper text might well be unfamiliar to other 
Francophone learners in the group. Similarly, if a student 
interested in biology read a piece about Nova Scotia 
fisheries and was curious about the meaning of crustacean, 
then other science majors who opted to read the same 
article might well be wondering about it too. If the pool of 
word findings was large, the chances that each student 
would find a useful body of new material to study would be 
increased. Thus, the technology design challenge was to 
offer students a simple format for creating their own course 
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by building up a large collection of vocabulary findings and 
to provide them with an easy way to share the collection 
with each other. Clearly, computer technology in the form of 
a collaborative on-line database offered a solution. 

The collaborative project we opted for was an on-line Word 
Bank. Figure 1 shows the homepage which the second 
author designed for the course. The button for word bank 
entry appears at the top of the middle column under Focus 
Activities. Clicking on this button brings up a data entry 
template which presents the student with spaces for 
entering a word, followed by an example of the word used in 
context, word class information, a dictionary definition, and 
the contributor's name. Each week the group of 33 students 
were required to enter five new words they had encountered 
in their newspaper reading in the Focus Word Bank. A 
sample of the 165 entries made in Weeks 1 and 2 of the 
course is shown in Figure 2. 
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a ESL • Academic VocabufcMy Microsoft Internet Explorer Y* 




ESL 298 

Academic Vocabulary Development 
• Instructor: Marlise Horst 



UWLt.. 

Activities 



Study UWL words at 
Lexical Tutor 



See last week's Quli 



Course FAQ's 

^.SP'CUSVu; 

Activities 



Enter Focus words 
into Focus Word 
Bank 



Previous Focus 
Wk 1*2 .Wk 3, 
Wk5.Wk6.Wk9, 
Wk 10 



Read the Focus 
section of the Globe 
and Mail 



Specialist 

Activities 



Enter specialist 
words Into Specialist 
Word Bank 



Previous Spedai 
Wk 4, Wk 8 



Get a 

feedbock 

form 



Figure 1 : Homepage for Academic Vocabulary Development, an 
experimental ESL course 
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She teaches a junior 
class. 


Adj 


1. (person) younger, lower 
in rank, than another; 2. 
(US schools and colleges) 
student in his third year (of 
four) 


FOCUS 


Guang 


There would be 
another husband , 
several suicide 
attempts , drinking, 
kited cheques ... 


Adj 


To get money or credit by 
using commercial paper of 
questionable value 


FOCUS 


Linndy 


this blouse is 
knitted, my grandma 
gave me it the last 
Christmas 


Adj 


(pr) knit to form or fabric 
from yam or thread 


FOCUS 


Catalina Uribe 


Lack of patients is 
not the problem, lt't 
the opposite. 


Noun 


To be wanting or missing. 


FOCUS 


Annie Derome 1 


1. Live up to a 
person'sexpections; 2. 
1 found it hard to love 
up to my ideals; 


Verb 


put into practice; - fulfil; - 
be equal to; - 


FOCUS 


Ren Wei He | 


I have to lobby to 
get the test done, 
lobby to get her a 
quick appointment 
with the gynecologist. 


Verb 


to try to influmce public 
officials and esp. 


FOCUS 


hao 


I have to personally 
lobby every day to 
get what my patients 
should just 
automatically get. 


Verb 


seek to persuade (an M.P.) 
to support one's cause. 


FOCUS 


stephane w. 


Does the mere 
presence of a golf ball 
create some sort of 


Adj 


Being nothing more than; 
Simple 


FOCUS 


Annie Derome 



Figure 2: Sample entries to the collaborative on-line database, the 
Focus Word Bank 



So far, our rationale for a collaborative on-line databank 
has emphasized advantages such as the efficient 
dissemination of a large body of material and the potential 
for individualizing instruction. Another important benefit, 
however, is the word learning that is likely to happen when 
a student creates a Word Bank entry. Hulstijn, Hollander 
and Greidanus (1996) have shown that the act of looking up 
a word in a dictionary increases the chances that the 
learner will remember it; it seems likely that the act of 
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typing out a definition and example sentence also 
contributes to making the new word memorable. 



In addition to the on-line Word Bank, an on-line dictionary 
and concordancer were made available to support learners' 
vocabulary learning. These computer tools will be described 
in a later section. 

Recognize that studying domain-specific vocabulary is 
important 

In the interest of living up to its name (Academic 
Vocabulary Development), we felt the course should offer 
participants the opportunity to study words specific to their 
chosen fields of study. We assumed that the idea of learning 
new science, economics, or art history vocabulaiy would 
appeal to students and it was clear that on-line technology 
could help address a range of individual interests. That is, 
the tools used to build a class word bank for general 
newspaper reading could also be used to build mini word 
banks in specialist domains. 

To implement this idea, we divided the students into special 
interest groups according to the academic field they 
intended to study. Five groups were formed around the 
following domains: arts, business, computers, science, and 
education. Each student was asked to locate a short reading 
on a topic in their field to share with other group members. 
Three times during the 13-week course (that is, once a 
month) students read two of these readings, summarized 
them, and entered five new words into the Specialist Word 
Bank, just as they did on a weekly basis with their 
newspaper reading. The entry form for this task can be 
accessed from the course home page by clicking on the 
button at the top of the third column entitled "Specialist 
Activities" (see Figure 1). The specialist data base allowed 
users to group word entries in alphabetical order, by 
student contributor, or by specific domain. Figure 3 shows 
a sample of Specialist Word Bank entries grouped by 
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domain: here we see items contributed by members of the 
Business Group. 
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breadwinner 


Mum’s the bread- 
winner in our family. 


Noun 


person whose eamtngs 
support his or her familly 


Business 


Amy Lin 




3 


peculiar 


[My keys have 
disappeared-Ifs most 
peculiar) 


Adj 


odd, strange, eccentnc, 
unwell 


Business 


Amv Lm 






outnumber 


The demonstrators 
were outnumbered by 
! the police. 


Verb 


be more in number than (sb) 


Business 


! 

Amy Lin 




crook. 


i That used-car 
salesman is a real 
I crook. 


Noun 


person who is habitually 
dishonest 


Business 


Amy Lin 


1 


Q 


Stop moping! 


Verb 


feel very unhappy and pity 
oneself 


Business 


Amy Lin 


J 




We bought to the 
tune of $15.1 billion in 
| 1985. 


Noun 


amount 


Business 


Annie 

Derome 




QyHI 


I have a 20% rebate 
,at Pharmaprix because 
j 1 work there. 


Noun 


discount, deduction 


Business 


Annie 

Derome 




1 seal 


) They affixed an 
[official seal and the 
i governors signature. 


Noun 


The impression made by 
stamping with such a 
device. Official mark. 


Business 


Annie 

Derome 




| outlay 


j Transportation is the 
j next largest outlay... 


Noun 


expenditure, cost 


Business 


Annie 

Derome 




fundraising 


! *we were fortunate 
[with Alexander Cossio 
'because he had done 
[fundraising before'. 


Noun 


An event held to generate 
such financial support. 


Business 


Annie 

Derome 




redeemable 


The bonds pay 6 % 
interest and are 
redeemable in three 
years. 


Verb 


1. compensate for the fault. 

2. buy back, exchange 
goods. 


Business 


Catalina 

Duque 






1 n tare s t, pay ment s. r-| 




a loan for t h e p unc h a se jo f 






a 


to 


fp'.. v.v ; 




CJw- 


[^Internet 5 




li 



Figure 3: Sample entries to the collaborative on-line database, 
Specialist Word Bank 
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Recognize that knowing sub-technical academic 
vocabulary is crucial 

Perhaps one of the most useful research findings to come 
out of corpus analyses of English texts is the identification 
of a core set of about 850 word families that occur 
frequently and consistently in academic texts - across 
disciplines. (A word family is defined as a root word, e.g. 
produce, and its derived forms, e.g. product, production, 
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unproductive, etc..) The importance of being able to 

at th? 1 ?? thC of word families on what is known 

rfr a lt U TT ty W ° rd List (Xue & Nation > 1984) is made 

naSSrt ^ C / ear m versions of ^ authentic textbook 
passage about increasing forest productivity shown below 

(from Nation, 1990). In the first version, all items that “ 
not among the 2000 most common word families of English 

eweSnce 5 Tfei Tt ! US ’ reading this Passage simulates 8 the 
xperience of ESL learners with no more than a basic 

knowledge of English when they are confronted with 
university texts. It is possible to get some sense of subject 

m th , iS version but m uch of the informational 
content is simply unavailable. 

Version 1 

....the increasing wood supplies will a larger 

force, an improved reading network, and 

— If the trees are 

then certain must be made. 



to be 



.„ . ; ' , *•“** iiiusi oe maae. 

y will include in: logging machinery and 

* I ntTOri n or J _xi. _ 



for the 



> logging trucks, and other 
products; .... 



of 



Version 2 

....the increasing wood supplies will require a larger 
labour force, an improved reading network, and 
expanded transport and processing facilities. If the 
trees are to be exported, then certain investments 
must be made. They will include investments in- 
logging machinery and equipment; logging trucks! 

and other required for the transport of 

processed products; .... 



hLh h f Second version > the words that are shown are both 
high frequency words from the 2,000 list and University 
Word List (UWL) words. Reading this version simulates the 
experience of an ESL learner who comes to the 4 sk of 
academic reading armed with knowledge of all items on 
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both lists. Now only one blank remains: the text suddenly 
becomes comprehensible and the missing item (vehicle) can 
be readily guessed from context. Interestingly, the UWL 
items that make the difference are not technical words 
specific to the domain of forestiy but rather all-purpose or 
subtechnical words like require, labour, process and 
equipment. It is clear that university-bound learners stand 
to profit a great deal from studying this key set of items. 

The covering power of the UWL is further detailed in Table 
1, which is based on analyses by Nation and Waring (1997) 
and Sutarsyah, Nation and Kennedy (1994). The fourth line 
suggests that an ESL student who knows both the 2,000 
most frequent words of English and the items on the UWL 
will understand 90% of the running words in a typical 
academic text, regardless of its subject matter. Since a 
receptive vocabulary size of about 3,000 words (2,000 + 
UWL = 2,800) has also been identified as the watershed 
between comprehension and non-comprehension in studies 
of academic reading by Laufer (1989, 1992), we felt there 
were compelling reasons to prioritize the study of the UWL 
in our experimental course. 



No. of word families 


Percent coverage 


Ratio unknown: known 


70 


50 


1:2 


1000 


75 


1:4 


2000 


80 


1:5 


2000 + UWL 1 


90 


1:10 


2000 + UWL + Specialist 


95 


1:20 


128,000 

A . l r- _ ... 


100 





words and coverage of academic texts 



The fifth line of Table 1 reflects findings by Sutarsyah, 
Nation and Kennedy (1994). Their work indicates that 
knowledge of several hundred words that recur frequently in 
the texts of a particular subject domain can offer additional 
coverage such that a reader who knew these would know as 
many as 95% of the running words. This suggested that the 
plan to devote some attention to domain specific words was 
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justified. Unfortunately, however, lists of high frequency 
core vocabulary for specific domains have not (yet) been 
compiled and so it was not possible to specify exactly which 
business, arts or science words were important for our 
learners to study. Nonetheless, we assumed the scheme 
outhned above for reading in subject domains and 
contributing new vocabulary to the Specialist Word Bank 

^?iHv re f. reSen ! 3 useful ste P 111 Preparing the learners for 
study in their chosen fields. 

To understand where the 35 students registered for the 
experimental course stood in relation to the word frequency 
zones identified as important for academic reading success 
we administered an updated version of the VocabularJ 
Levels Test (Nation, 1990; Schmitt, 2000). This instrument 
uses a multiple-choice cluster format to test receptive 
knowledge of items sampled from each of five zones: the 

i n™^ 0531 frec * uent words; words on the 3000, 5000 and 
10,000 most frequent lists; and the Academic Word List (a 
recent update of the UWL; Coxhead, 2000) The test 
requires testees to match items to simply worded 
toons. An example of a question cluster from the 
section that tests the Academic Word List is shown in Table 



1. benefit 

2. labour 


work 


3. percent 


part of 100 


4. principle 


generated idea used to guide 


5. source 


one's actions 


6. survey 





Table 2: Sample question from the AWL 
Levels Test: Version 1 (Schmitt, 2000) 



section of the Vocabulary 



IS h!? UltS u conl ' ir ™ ed 1116 expectation that in terms of 
ocabulaiy knowledge, the learners were indeed a very 
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diverse group. Although the group means in Table 3 suggest 
that students could recognize the meanings of 80% of the 
tested words at the 2,000, 3,000 and Academic levels, the 
standard deviations reveal that individuals varied 
considerably. Some of the French speaking students scored 
high on the Academic list (which contains many words of 
Latin origin) but low on the 2,000 list (which contains many 
words of Germanic origin). Many of the Asian students 
(mostly Chinese speakers) scored high on the test of the 
2,000 list but fared less well on other lists. 





Pretest/ 30 
(n=28) 


2000 




M 


25.27 


SD 


4.46 


3000 




M 


24.61 


SD 


3.96 


5000 




M 


20.86 


SD 


4.84 


10,000 




M 


9.68 


SD 


5.21 


Academic 




M 


24.77 


SD 


3.94 



Table 3: Pre-test means on the Vocabulary Levels Test by section; 
maximum score = 30 



These outcomes confirmed our intuitions about the need for 
an individualized course designed to meet highly diverse 
needs. Furthermore, it was clear that the students had 
plenty of work to do in the Academic/ University Word List 
zone. Few students had full masteiy of words at this level, 
and given its importance for academic reading success, it 
seemed likely that all would benefit from intensive study of 
these words. To implement this goal, we divided the 800 
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words of the UWL into twelve 65-word lists for week-by- 
week studying and testing. Thus, in addition to reading and 
summarizing two readings each week and submitting Word 
Bank entries, students prepared for a UWL quiz. About 30 
minutes of each of the twice-weekly 90-minute class periods 
were spent either on UWL learning activities or testing. An 
example of a weekly quiz appears in Appendix 1, and the 
self-access version used by students following the written 
test appears in Appendix 2. 

Raise awareness of proven word learning strategies 

So far we have discussed the vocabulaiy learning materials 
on offer (the Word Bank entries and the UWL), ways of 
delivering it (class and website activities) and an evaluation 
component (the UWL quizzes). However, as O'Dell (1997) 
and others point out, training students how they can learn 
most effectively is a key aspect of any language course. Our 
examination of the vocabulaiy learning research identified 
five main strategies for successful retention of form- 
meaning associations. These are: keyword mnemonics 
(Brown & Perry, 1991) word-part analysis (Sokmen, 1997), 
elaborative sentences (Brown & Periy, 1991; Ellis 1997), 
dictionary use (Hulstijn, Hollander & Greidanus, 1996) and 
concordancing (Cobb, 1997). We set the goal of familiarizing 
students with each of these in class activities and, where 
possible, on the website. 

In practice, it turned out that some of the strategies were 
limited in their applicability. For instance, the much 
acclaimed keyword imaging technique can only be applied if 
an English word to be learned sounds like an LI word and 
represents a concept that can be pictured. Thus, one 
student was able to draw a picture of a disgusted teacher 
throwing failed papers into the air to remember the English 
word flunk. Since flunk sounds like the French word 
flanquer (throw, fling) the vivid papers-in-the-air image 
creates a strong link to the new word and its meaning. 
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However, it is clear that many words, especially abstract 
ones, do not lend themselves to this treatment. 

Two strategies that could be applied to any new word were 
consistently supported in class activities throughout the 
course. These were dictionary use and concordancing. 
Support activities for dictionary use included exercises in 
identifying correct definitions of words that have different 
senses in different contexts, and comparing dictionaries 
designed for native speakers to those designed for learners. 
Students were also shown how to access an on-line 
dictionary via the Lexical Tutor button in the first column of 
the home page (see Figure 1). 

The Lexical Tutor button also offered learners an easy-to- 
use on-line concordancing tool (developed by Chris Greaves 
of the Hong Kong Polytechnic University). A concordancer 
searches a large body of text to find every occurrence of a 
particular word or phrase and displays these in a format 
that allows the user to see the many different instances of 
the word in use. A concordance for the word abandon 
consisting of 13 instances of the word in use is shown in 
Figure 4 (on following page). In principle, a concordance 
should be a powerful resource for learning. Because the 
learner can examine a number of sentences containing the 
new word, chances are that he or she will meet at least one 
that is easy to understand. If the learner engages in solving 
the puzzle (i.e. guessing the word's meaning), the 
concordance offers the opportunity to test a solution in 
other sentences. Research by Cobb (1997; 1999) has 
confirmed the usefulness of learning by concordancing. He 
found that learners who examined concordances were more 
able to transfer new word knowledge to novel contexts than 
learners who studied definitions. In the experimental course 
described in this paper, students were shown on the first 
day how to use the concordancing tool and then again at 
several times later. In addition, classroom concordancing 
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activities on paper were designed to raise student 
awareness of this strategy. 



Challenge learners to study hundreds of new words 

Researchers differ on the number of words language 
learners need to know to readily comprehend textbooks 
used in university content courses. Research by Laufer 
(1989, 1992) points to a minimum receptive vocabulary size 
of 3000 high-frequency word families. Work by Hazenburg 
and Hulstijn (1996) suggests that the figure may in fact be 
higher than this but the main message of these studies is 
clear: the university-bound learner needs to know 

thousands of words. For many learners, this means 
acquiring hundreds if not thousands in fairly short order 
in a semester or two of ESL study, if possible. Thus there 
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are compelling reasons to make acquiring knowledge of 
hundreds of new words an explicit course goal, even if it 
means encouraging learners to resort to studying lists. 
Rote memorization tasks are out of fashion in 
language teaching, but Nation (1982) suggests that 
currently popular methodologies may be neglecting a 
powerful learning technique. He points to memory 
experiments where participants have been found able to 
learn (and retain) as many as 50 new word and translation- 
equivalent pairs an hour. Admittedly, this kind of memory 
work cannot be expected to result in full knowledge such 
that a learner understands all the senses of . new words or is 
able to use them correctly in elegant original sentences. 
However, we reasoned that some initial engagement - albeit 
incomplete - with over a hundred words each week was 
potentially more useful to university-bound learners than 
intensive study of the dozen or so items that is more typical 
of ESL courses. 

Unlike the simple word/ translation pairs used in the 
memory experiments, the on-line lists available for study in 
our experimental course were richly informative. In addition 
to offering the definition of a word and its part of speech, 
each Word Bank entry provided an example of the word in 
use. Since these sentence contexts generally came from 
material students had read, we assumed that many entries 
would also provide memory links to class discussions of 
newspaper articles, summary writing, and other activities. 
The on-line concordancing tool meant that students could 
also study UWL items in a wide variety of sentence contexts. 
To motivate students to study large numbers of words, we 
included two exams in the plan for the course, a mid-term 
and a final. For each of these, students were expected to 
study 400 UWL items, 200 items from the newspaper Word 
Bank, and 50 Specialist Word Bank items in their particular 
subject area. 
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Research questions 



Our investigation of the experimental course focuses on two 
key issues: the usefulness of the on-line resources, and the 
amount of new vocabulary knowledge gained by the learners 
who used them. One way of examining the usefulness of the 
learning materials is to consider their quality. Indeed the 
claim that learning vocabulary with a collaborative on-line 
database is effective rests on showing that students were 
indeed generating accurate and useful materials for their 
own learning. We were also interested to see whether the 
quality of the entries changed as students gained experience 
in working with dictionaries and selecting examples during 
the 13-week course. Thus the first research questions we 
consider are as follows. 

1. What was the quality of student-produced on-line course 
material (the Word Bank entries), and did entries improve 
over time? 

We have argued that an important advantage of a 
collaborative on-line project is its potential to offer 
instruction tailored to individual needs. Since the pre- 
testing had shown that needs were indeed highly diverse, 
we were interested in seeing whether different kinds of 
learners were using the resource in different ways. Two very 
distinct constituencies in the group were learners of Asian 
and Romance language backgrounds. Romance language 
speakers are able to exploit cognate knowledge for clues to 
the meanings of the many English words of Latin and Greek 
origin, a strategy that is not available to Asian language 
speakers. Thus, as Laufer (1997) points out, words of Latin 
origin like perspective or anticipate may look opaque to one 
learner but totally transparent to another. Exploring how 
these two groups of learners used the on-line database 
seemed likely to be a useful initial indicator of how well the 
on-line resources served the needs of different types of 
learners. This prompted the following research question. 
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2. Did students of Asian and Romance language 
background enter different types of words? 

The remaining questions address the important issues of 
amounts of new word knowledge gained in the experimental 
course and factors that might explain growth results. 

3. Did learners increase their vocabulary knowledge? 

4. Which strategies (e.g. dictionary use) were associated 
with learning gains? 



Participants 



The 33 students who registered for the experimental 
vocabulary course at Concordia University (Montreal) 
represented a variety of first language backgrounds. About 
two thirds of the group were speakers of Asian languages 
(Chinese and Vietnamese) and about one third had 
Romance language backgrounds (Quebec French, Spanish 
or Portuguese). There was also an Arabic speaker and a 
Farsi speaker in the group. All had been assessed as having 
minimal or inadequate proficiency for university studies on 
a placement test administered by the institution. There was 
a range of abilities in the group but they can be generally 
termed intermediate-level learners. Most had been admitted 
to the university on condition that they take courses to 
improve their English. 



Results 



Word Bank entries - the quality question 

We used a ratings procedure to investigate the quality of 
Word Bank entries (research question 1). To evaluate 
students' example sentences, we began by selecting at 
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random 40 sentences entered during the second week of the 
course. Forty more sentences entered during the eleventh 
week were also selected so that early and late entries could 
be compared. Next, following a method devised by Beck, 
McKeown and McCaslin (1983), we deleted the target words 
and asked a native speaker of English to draw on 
information in the sentences to supply the missing items. 
The sentences were then evaluated using the following 
scheme. If the rater's guess matched or nearly matched the 
word in the original sentence, the sentence was awarded a 
score of 4. If a guess showed general similarity to the 
missing word, the context was considered to be supportive 
and was awarded a 3. An example of such a sentence 
appears in Table 4 where we see that the rater supplied 
anticipating instead of the original term yearning in the 

example sentence "I was this trip". Sentences prompting 

guesses that bore little resemblance to the target words 
were considered neutral and were awarded a score of 2. 
Misleading contexts scored 1 point. See Table 4 for 
examples. 



Student data base entry 


Informant's guess 


Rating 


The theatre has a seating 
capacity of 800 


The theatre has a seating 
capacity of 800. 


4 

exact 


Preminger cronned Jean's 
hair. 


Preminger cijt Jean s hair 


4 

near exact 


I was vearnine this trip 


I was anticipating this trip. 


3 

supportive 


He commit himself in 
writing this book 


He excelled himself in 
writing this book 


2 

neutral 


Her religions-minded 
parents had met at a 
science convention. 


Her reUgions-shunning 
parents had met at a 
science convention. 


1 

misleading 



Table 4: Rating scheme for assessing quality of context entries 



To rate the quality of definitions entered in the Word Bank, 
we selected 40 definitions at random from weeks 2 and 1 1 
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of the course. Again, we used a 4-point rating scheme. 
Definitions that were simply and clearly worded and 
matched the sense intended in example sentences were 
awarded the full mark of 4. Definitions that were accurate 
but contained difficult language were awarded a 3. The 
wording problem is evident in the case of chutney, where the 
definition "pungent condiment made of vinegar and fruits" 
is clearly accurate but of doubtful usefulness because of the 
potential difficulty of the words pungent and condiment. 
Uninformative definitions such as the circular one for 
mythical shown in Table 5 were rated 2 points while 
definitions that did not match the sense of the example 
were awarded 1 point. 



Word 


Student data base entry 


Rating 


She mourned for 
her dead son. 


to have or show great 
sorrow, usually for a 
person who has died 


4 accurate, easy to 
understand 


I enjoy chutnev 
with my turkey. 


pungent condiment made 
of vinegar and fruits 


3 accurate, hard to 
understand 


I was ve amine this 
trip. 


I was anticipating this 
trip. 


3 supportive 


Arthur and 
Mordred are 
mythical persons 


of or existing in myth 


2 circular or too long 


You'd think Alberta 
would be bristling 
with warnings to 
Ottawa. 


Thick strong animal hair 
used to make brushes. 


1 inappropriate 
sense 



Table 5: Rating scheme for assessing quality of definition entries 



Mean scores indicated that the overall quality of context 
sentences was fairly high (Table 6). The mean rating for 
early contexts amounted to 2.7 while the mean for later 
contexts was 3.0 (out of 4). That is, late entries earned the 
score assigned to supportive sentences, and the early entries 
closely approached this level. Thus we can conclude that in 
general, the student-generated material offered useful 
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context information about new words. These ratings results 
are higher than the 2.5 mean ratings other studies using 
this methodology have found for natural texts (Horst, 2000; 
Zahar, Cobb & Spada, in press), so it is possible to conclude 
that the student entries succeeded in being more 
informative than ordinary sentences would be. The higher 
mean rating for the later data suggests that students 
improved the quality of their entries as the course 
progressed. Although, a t-test for independent samples 
showed that the gain was not statistically significant, it 
seems reasonable to assume that the rising profile would 
continue with more time and practice. 





Week 1 


Week 11 


M 


2.70 


3.00 


SD 


1.02 


1.04 



Table 6: Quality-of-context ratings (n = 40) 

Definitions also appeared to be of a mainly high quality 
(Table 7). In fact, liie definition results are similar to the 
sentence findings. The mean ratings of around 3 (the score 
awarded to accurate definitions with wording difficulties) at 
both the begi nning and end of the course suggest that the 
definitions were generally accurate throughout. Again, the 
data suggest that the quality of definitions improved during 
, the course. Though the difference was not found to be 
statistically significant, it seems likely that definitions 
would continue to improve over the longer term. 





Week 1 


Week 11 * 1 


M 


2.95 


3.15 


SD 


.93 


1.08 - — 1 



Table 7: Quality-of-definition ratings - 7 

In general, we can conclude that both the example 
sentences and the definitions students supplied for each 
other in the Word Bank project were of very satisfactory 
quality; ~ Interestingly, ~a ~few~ students- complained r about — 
occasional spelling or grammar errors they spotted in the 
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entries, but none complained about the word information 
on offer. Our analysis confirms that the information 
provided in the entries tended to be useful and accurate. 
However, some qualifications are in order: the standard 
deviations suggest that quality was rather inconsistent, and 
the high but not perfect average scores suggest that 
students may benefit from training in how to produce clear 
definitions and supportive example sentences. 

Word Bank entries - the individual differences question 

To investigate whether students of different LI backgrounds 
were using the on-line resources to meet varying vocabulary 
needs (research question 2), we took a close look at words 
entered by students in two distinct groups: Asian and 
Romance language speakers. To compare the words that 
learners in the two groups looked up, we prepared two 
corpora of 300 words each. The Asian corpus consisted of 
the 300 items entered in the Focus Word Bank during the 
first three weeks of the course by learners whose first 
language was Chinese or Vietnamese. The Romance corpus 
consisted of the 300 items entered by French, Spanish and 
Portuguese speakers. Each corpus was analyzed using 
HyperV ocabProfile (Cobb, 1998; based on Huang &Nation, 
1998), a computer program that groups English words into 
frequency bands. That is, the program allowed us to see the 
extent to which students in the two groups looked up 
common and less common words. Of special interest were 
the number of look-ups in the UWL band (which contains 
many words of Greco-Latin origin). We hypothesized that 
the proportion of lookups in this zone would be larger in the 
Asian group than in Romance group. 
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■ Romance 



0-1000 1000- UWL Off-Ust 



2000 

Frequency zones 



Figure 5: Distributions of 300 looked-up words in two LI -based 
groups (figures in percentages) 

The results shown in Figure 5 are striking. If we consider 
the high frequency categories (the 0-1000 and 1000-2000 
most frequent bands), we see that 12% (7 + 5) of Asian look- 
ups were common English words. However, this category 
accounts for a far greater proportion of the Romance look 
ups; in fact, over a quarter (18 + 9 = 27%) of all the words 
looked up by Romance speakers were in this zone. A 
possible explanation is the fact that the 0-1,000 and 1,000- 
2,000 bands contain a high proportion of words of Anglo- 
Saxon origin, words which have no cognate equivalents in 
Romance languages and are therefore more likely to be 
unfamiliar to Romance speakers than Latin-based English 
words. The occurrence of common words of Germanic origin 
like flew, storm and height on the Romance list suggest that 
this was the case. The notion that learners in the Romance 
group directed their attention to non-cognates is also 
confirmed by the third column of data where we see that 
these learners looked up fewer of the Greek and Latin based 
UWL items than the Asian learners, for whom these words 
appear to be difficult.— In summary, it is clear that the two — 
groups were looking up different types of words, and there 
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is reason to think that both groups were well served by a 
course designed to address individual vocabulary needs. 

Vocabulary learning - the growth question 

To determine how much students had learned as a result of 
taking the course and participating in the collaborative 
Word Bank project (research question 3), we measured 
students' receptive vocabulary sizes at the beginning and 
end of the course by administering updated versions of the 
Vocabulary Levels Test (Schmitt, 2000; Schmitt & Schmitt, 
forthcoming). As discussed earlier, this instrument is 
designed to assess receptive knowledge of words sampled 
from lists of the 2,000, 3,000, 5,000 and 10,000 most 
common words of English and the Academic Word List (a 
list similar to the UWL). Vocabulaiy learning gains were 
determined by calculating the differences between learners' 
pre- and post-test scores. 

Pre-post results and gains are shown in Table 8. The 
maximum score possible in each section of the test was 30. 
While the general picture is largely one of growth, it is also 
evident that some of the changes are very small. Statistical 
analysis (ANOVA and post hoc t-test) showed that mean 
scores on the Academic Word List section differed 
significantly (t = 2.62; p < .05). Although the gain of about 
two new words in this category may appear rather minor, if 
we extrapolate this result to the entire word list, we see that 
learners achieved a substantial amount of growth. The gain 
of 1.73 words in 30 represents a growth rate of 5.8%; when 
this figure is applied to all 800 words on the UWL, we arrive 
at a gain figure of about 46 new words (.058 x 800 = 46.13). 
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Pretest/ 30 


Posttest/ 30 


Gain 


3,000 








M 


24.61 


24.86 


.25 


SD 


3.96 


3.37 I 


3.53 


5,000 








M 


20.86 


20.77 


-.07 


SD 


4.84 


5.57 


4.36 


10,000 








M 


9.68 


9.77 


.11 


SD 


5.21 


6.52 


5.08 


Academic 








M 


24.77 


26.50 


1.73 


SD 


3.94 


3.63 


3.09 



Table 8: Pre-and post-test means: Vocabulary Levels Test (n = 28) 

Clearly, the learners acquired new receptive vocabulary 
knowledge as a result of studying in the experimental 
course. As we have seen, increases in knowledge of items on 
the Academic/University Word List accounted for most of 
the growth. Since these sub-technical terms are important 
for university ESL learners to know, we can conclude that 
the course achieved an important objective. However, 
increased knowledge of UWL items is hardly surprising 
given the amount of attention paid to the UWL in class. 
Every week students participated in activities to support 
UWL word learning and studied for a weekly quiz; students 
studied these items again for midterm and final tests. 

This leaves unanswered the question of why evidence of 
growth was so slight in non-UWL zones -. .zones that the 
Word Bank activities were designed to address. One 
probable explanation is that the Vocabulary Levels Test was 
not sensitive enough to capture the incomplete but real 
knowledge that a learner might retain from the experience 
of reading an on-line definition of a word and a single 
illustrating example. Work by Horst (2000) has shown that 
the learning impact of one or two encounters with a new 
word can be captured but that very sensitive measures are 
required. Additionally, the sampling technique used to 
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construct the Vocabulary Levels Test is problematic for the 
assessment of low-frequency words. For instance, a learner 
who acquired new knowledge of a word in the 5,000-10 000 
most frequent band through studying the Word Bank is 
highly unlikely to encounter that word on a test that 
samples only 30 items of the 5,000 words in the band. 
Thus, there is no reason to conclude that students did not 
protit from the on-line collaborative activities; it is highly 
probable that they did. Rather, the results point to the 

importance of using sensitive measures to assess 
vocabulary learning. 



Another explanation for slight growth outside the UWL is 
that the vocabularies of specific academic disciplines 
remain as yet undefined, so that it is not possible to target 
the characterizing terms of economics, stage design, or any 
other discipline for purposes of either teaching or testing 
Our students may well have covered much of the lexical 
temtoiy of their chosen domains in their Specialist Groups 
actmties, but we have no measure for establishing this 
other than the 10,000 level of the Levels Test. 

Keys to success - the strategies question 

Although we familiarized students with a variety of proven 
strategies for learning vocabulary in the course, we limited 

** met *"> 1) strategies 

that could be applied to any word (see the discussion of 
keywords above for an instance of a limited strategy); and 2) 
strategies that students were familiar with from the first day 
of the course onwards (i.e. there was ample opportunity to 
use the strategy). Two traditional strategies (using a 
monolingual or a bilingual dictionary), and two 
computerized strategies (using an on-line dictionary and 
using a concordance) met these criteria. 

We used a questionnaire attached to the Vocabulary Levels 
post-test administered in the final week of the course to 
explore the extent to which students made use of the 
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various strategies. The questionnaire asked students to rate 
their use of each according to the following five-point scale. 

1 = never 

2 = once or twice 

3 = fairly often 

4 = very often 

5 = almost always 

Then, to determine which strategy was most closely 
associated with learning gains, we entered student ratings 
of the four strategies into a multiple-regression analysis 
with pre-/post Academic Word List gain scores as the 
dependent variable. 

Traditional dictionary use was clearly more widespread in 
the group than use of the computer dictionary and 
concordancing tools. Figure 6 shows the mean ratings of 
about 3.5 for using bilingual dictionaries (e.g. English- 
Chinese) and monolingual dictionaries (English-English); in 
other words, these resources were both used often. Ratings 
for the non-traditional tools are lower; students appeared to 
prefer using the on-line dictionary to concordancing. 




Bilingual Monolingual Computer 

dictionary dictionary dicctionary 

Strategy type 



Concordance 



Figure 6: Strategy use (n = 22) 



The regression analysis revealed that the strongest predictor 
of vocabulary learning outcomes was monolingual 
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dictionary use. However, the use of this strategy is 
negatively associated with learning gains. The most 
reasonable interpretation of this finding is that good 
learners who already knew many words on the UWL (and 
therefore had little opportunity to register gains on the test) 
were consistent users of English-English dictionaries. A 
more interesting finding appears in the second line of Table 
9 where we see that of the remaining predictors 
concordance use is most strongly associated with 
vocabulary learning gains (r = .38) albeit with a probability 
level that suggests some role for chance effects. This finding 
suggests that even though average concordance use ranged 
between the never and almost ratings in the group as a 
whole, students who did use this feature were likely to 
experience vocabulary gains. 



Strategy 


Correlation 


Probability 


Monolingual 


-.46 


.03 


Concordance 


.38 


.08 


Bilingual 


.14 


.55 


On-line 


.05 


.82 



Table 9: Summary of regression analysis for variables predicting 
academic vocabulary gains (n = 22) 



Conclusion 



We are convinced that the collaborative database is a 
valuable tool for vocabulary acquisition for learners who 
have moved beyond the elementary level. The technology is 
clearly able to absorb the variety of lexical needs that 
characterize learning at this level. Our students have shown 
that they are willing to use the tools we have developed, 
that they use them reasonably well, and that they learn 
some words by using them. How many words, we can not 
say as yet, for reasons mainly related to insensitive or non- 
existent vocabulary measures. Perhaps the lack of suitable 
tests for an experiment such as ours is not surprising. Few 




30 



218 T om Cobb and Marlise Horst 



instructional designs in the past have attempted to 
introduce new words in the numbers we have targeted, so it 
is reasonable there might be few suitable ways of measuring 
our results. The 5,000 and 10,000 level of the Levels Test is 
clearly a blunt instrument. Tests that can measure fine 
degrees of knowledge simply, and that target the 
vocabularies of specific domains, are urgently needed. 

We are presently feeding our discoveries from this 
experiment into a revamped vocabulary course with a 
collaborative database at its centre, and we expect to run 
this course again soon. At the same time, we want to 
provide a fully independent version of the course on the 
Internet to cater to a virtual clientele worldwide. In fact, our 
vocabulary learning tools were inadvertently left on the Web 
when our course had finished, and learners from various 
comers of the world have already started using these tools, 
adapting them to their own purposes! 
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Appendix 1 

UWL Quiz 8: linguistics-outcome 



Name: 



^ Write the number of the word next to its definition (12 
points): 



1. magic 

2. magnetic 

3. moist 

4. odd 

5. obsolete 

6. mature 



strange 

wet, damp 

old, out of date, useless 



1. margin 

2. orbit 

3. null 

4. navy 

5. motive 

6. momentum 

1. mobile 

2. maternal 

3. nuclear 

4. neutral 

5. normal 

6. logical 

1. litigation 

2. location 

3. migration 

4. notation 

5. orientation 

6. obligation 



reason to do something 

circular movement 

empty space at the edge 



choosing neither side 

showing clear thinking 

able to move 



movement, travel 

duty, responsibility 

way of writing 
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below to complete the passage (8 points): 

magnitude 

occupy 

obvious 



Barings Bank used to be one of the oldest and most respected 
British investment banks. It had branches all over the world and manv 

famous customers including the British (1) Queen 

Elizabeth. But in February 1996 there was bad news at the Singapore 
branch. In fact, it looked like Barings was in serious trouble. At first the 

* 2 )" of the problem was not clear. Nobody knew for 

sure how many bad investments had been made or how much money was 

but il sotm became (3) that the losses were over 

$1.3 billion dollars, and so large that the 232-year-old bank was forced to 
close with great losses to its customers. 



How could such a disaster (4) ? How was it possible 

such a respected and trusted institution to have made such mistakes? 
The top management of Barings promised a thorough investigation and 

they soon found out who was responsible: a young trader called Nick 
Leeson. 



B, Cloze. Choose from the words 

outcome occur 

monarch nutrients 

maintain luxuries 

The Man Who Broke the Bank 



Here is his story: Leeson had done very well at Barings and had 
received huge bonuses and rapid promotions for his excellent 
performance. He came from a very ordinary working-class English family 
mid he and his wife enjoyed their new life and the 

that came with wealth and success in 

Singapore. They ate at the finest restaurants and played tennis at the 

best club. Leeson was determined to (6) his 

record of success at the bank. 



His method was simple. He made very large, very risky 
investments for Barings in the hopes that there would be enormous 
profits. If there were losses, he entered them in a secret account and 
hoped to pay off the growing debt with profits from the next investment 
success. But the debts increased so fast that Leeson lost control. By the 
time the fraud was discovered, it was too late to save the bank. 



was 
a 



the 



Barings Bank was forced to close, but what 

* f° r Nick Leeson? He was given a prison 

sentence of just six and a half years for his crime, and he did not se em to 

be very sorry about what he did. How did he (8 

his time m prison? He spent most of his time writing. Recently, he 
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published a book called Rogue Trader— How I Brought Down Barings Bank 
ppd Shook the Financial World. Buy it and read it if you like, but 
remember: Your purchase is helping to pay the legal bills of a thief. 

- from the Toronto Globe & Mail, March 1996 




38 



I 



226 Tom Cobb and Marlisc Horst 

Appendix 2 

Tests were placed on-line for students to check immediately 
after completion, using an authoring script developed by 
Chris Greaves. The fill-in words can be clicked to generate 
concordances. 



3 Web OozeMalci Script Wrz<nd • Mtciosoff Internet Exploits 




ggpg 



m 



UWL Level, Cloze 8 



[occupy j magnitude [luxuries joccur 



f ma intain i; obvious j [monarch { o u tcom e 



Enter your. name : \ 

Tho ^n^Whd Broke thG Bankri: ' 

BaringsBahkused to:be icma^of the ddestrand^rnost respected British investment banks. 

It had branch es atj over the' world: arid many famous customers, including the British. ( 1 

I... Queen Elizabeth; But last year in February there was bad riews at the 

Singapore bra inch. In fact, it looked Kke Barings was in serious trouble. At first the (2 

I llj.pf the problem was not dear. Nobody -knew for sure how many bad 

investments had been made or; how much money was involved, but it.soon became (3 

I U that the, losses were over $1.3j billion dollars, and so large that the 232- 

year-old bank was forced to close with gre at losses to its customers, ' .y : 

How could such a disaster (4 1 g? How was it possible for such a respected* and 

tres ted institution to have made su^ mwtalresf The -top mwagement of Barings promised 
a ‘thorough investigation andthey scon found out who was responsible: a young trader . 
cajJjtt.Nfck-^ if ‘ -v :•/ . 

Hereis history: La a son had[d(^;y^^^l at; BaringsT,aind h ad received huge ^bonuses* ‘ 
andrap/d; promotions forhls excellent performance. Ha came "from a Very ordiriaryi-, 
working-class : EnqGsh famHy.j'an^ : hei and : hijT;^fe; enjoyed their new lift 'and. 1 ?' 

I Jdi&hat "came;; With wealthand success in Singapore , They ate at : the firiest J 

restaurants a nd played tennis at the best dub. ^eson becarnQ ;det ermine d; to (6 ~ \ 

I Mws.tecgrd \’ ^ V *•: .7 ? 1 

His method wa^sii^fHe made V^rylia^^veiy^ ^ risky investments for Banrigs ortKe^r'- 
hopes tha t there would . be ehorrnous prefitsr lf, there were losses, he entered them in 






1 1 

i 



BEST COPY AVAILABLE 



39 




<1) 

O) 

8 . 

d) 

sz 



E 

0 

1 
0) 
x: 

"co 



.O) 

(0 

T3 

C 

CO 

CO 



Q. 

0 

1 
£ 
O) 

c 

? 

2 
o 

5 

o 

LU 

2 

o 

* 

o 

LU 

X 

o 

a) 

CO 

$ 

CL 



3 

O 

o 

*a 

8 



£ 

a) 

£ 

<D 

s 

1 

t 

o 

§ 

*c 

2 

O) 

CO 

c 

o 

to 

m 



8 



http://www.cal.org/ericdl/Releaseform.html 






C3> O 

i CD 
£1 

II 

<D E 

CQ r- 



<D O 

al 

§ l 

o co 

-J.$o 

*o 

£ « 
® c 
■c o 

II 

6? 

Q. 

<D 





ERIC 




0 5 

4— • 

3-g 

1 « ^ 

Q.O g- 

S>E 8 

g 1 "® 

s CD S' 

li & 

fl) ^ 
a°c 

- CD CO 

®£rv 

WOO 
C3 7= 

|2o 




8 

8 

£ 

O 

.52 

X 

8 

k— 

CD 

.c 

CD 

c 

3 

-Q 

t 

c 

s 

o> 

52 

8 

3 

& 

O 



CO 

CO 



a> 

CL 



E 

8 

> 

JM 

aj 



C 

o 



1 

§■ 

8 

> 

2 

CL 

8 . 
W T “ 
£ © 
C S 



8-g 

$ CO 
m CO 

B CD <j) 

O 

O 

Q-o. 

If 

W CO 

Ec 



r? 5 

o 

o 

T3 



§ 




® 

C ® CD 

£>fc 8 

0)£ O. 



http://vAvw.cal.org/ericdl/Releaseform.htm I 



II. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or , if you wish ERIC to cite the availability of the document from another source, please provide the following 
information regarding the availability of the document. (ERIC will not announce a document unless it is publicly available, and a dependable source can be specified. 
Contributors should also be aware that ERIC selection criteria are significantly more stringent for documents that cannot be made available through EDRS). 



CO 



ERIC 




rr co 

QJ a> 

oi 

— -o 

o § 



CO 



If 

O s 

El 

CD 






*0 

> 

o 

Q. 



Oa, 

I 8 



W a) 

oc $ 
0. $ 
111 5 
CC « 

H I 
I§ 
(Dt 
E ® 
> ° 
il 

o 
o 



o 

co 

Oe 

H — 

CD 
j= 
CO 



o 

IT 

Ul 



a) 



c 

o 



o 

=3 



jI 

2* 

£E c 

QJ 2 

Li. ~ 
111 
QC 



CO 



£1 

a) 

'C 




O 

CC 

UJ 

a) 

'k* 

Q. 

O 

8: 

CD 

CD 

XT 



eg 

£ 

to 

E 

L. 

3 

2 



x: 

o 

x: 

$ 

a 

.CO 

O) 

c 



CO 

CO 

CD 

O) 

CO 

3 



5 



o 

CD 

CO 

3 

o 

x: 

05 

C 



So 

O LU 

LL ® 

CO 2 



oS 

k. 

UJ I 
CO ^ 

^ * 
O § 

h o 



U 




o 

o 

GC 

UJ 



o 

o 

o 

cvi 

8 i 1 

s s 

SI® 

s 5 

® <9 
<e ? 

CM 

O 

O 

00 



http://www.cal.org/ericdl/Releaseform.html 



